Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

automod: rules engine framework #434

Merged
merged 35 commits into from
Nov 20, 2023
Merged

automod: rules engine framework #434

merged 35 commits into from
Nov 20, 2023

Conversation

bnewbold
Copy link
Collaborator

@bnewbold bnewbold commented Nov 14, 2023

automod is a package to help write moderation automation "rules" to catch things like spam and flag accounts or content for human review (or potentially even auto-action).

hepa is a simple service which consumes from a BGS firehose and pushes events through automod.

This is an early draft of the rules engine, some notes:

  • rules are written in golang here. we might end up using a DSL or scripting language of some sort eventually
  • the framework and some example rules are here in a public repo. we'll obviously keep some rules and configuration private in the long run
  • intention is to use redis for caching and state persistence. will likely end up with additional datastores

}
collection := strings.SplitN(path, "/", 2)[0]

switch collection {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fyi: You can do a type switch in Go:

switch rec := rec.(type) {
case *appbsky.FeedPost:
  // inside this case, rec is the correct type
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I had it that way first, but need to verify collection/type mapping somehow (aka, type needs to be FeedPost and collection is "app.bsky.feed.post") and this felt more legible than switch on type and verify collection string.


// base type for events. events are both containers for data about the event itself (similar to an HTTP request type); aggregate results and state (counters, mod actions) to be persisted after all rules are run; and act as an API for additional network reads and operations.
type Event struct {
Engine *Engine
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it feels a bit odd to me that the event would carry the engine reference, seems like the 'persist' methods should just be on the engine itself and we should pass the event through that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, agree this feels a bit weird.

the rules reach back to the engine other ways as well though: helpers on the event can fetch counters and things. it might be more golang-idiomatic to pass both the engine and the event to all rules.

part of this might be poor naming. "events" here are kind of more like echo.Context, maybe "EventContext" would be better.

might still be a bad design pattern.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the more obvious/idiomatic thing would be to have the rules be methods on the engine, or methods on some context object in general. then you would access things like counters, logger, or other external resources through the parent struct. but i'm trying hard to make rules free-floating rules so they can be implemented in arbitrary packages and collected at configuration time.

@bnewbold bnewbold marked this pull request as ready for review November 20, 2023 07:58
@bnewbold bnewbold requested a review from ericvolp12 November 20, 2023 07:58
return nil
}

did, err := syntax.ParseDID(evt.Repo)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not now, but we should probably make lexgen use the syntax.DID type for fields that have format:did

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup! I can't remember if I have tested yet, but the syntax.DID type implements the text marshaling interface, so this should Just Work

Copy link
Collaborator

@whyrusleeping whyrusleeping left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like a good scaffold to start from. lets get it running and start poking at things.

@bnewbold bnewbold merged commit aea86b8 into main Nov 20, 2023
8 checks passed
@bnewbold bnewbold deleted the bnewbold/automod branch November 20, 2023 21:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants